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ABSTRACT 


A well-ordered traffic management system is required in all types of roads, 
such as off roads, highways, etc. There has been several laws and speed 
controlled measures are taken in all places with different perspectives. Also 
Speed limit may vary from road to road. So there are number of methods has 
been proposed using computer Vision and machine learning algorithms for 
object tracking. Here vehicles are recognized and detected from the videos 
that taken using surveillance camera. The aim is to identification of the 
vehicles and tracking using Haar Classifier, then determine the speed of the 
vehicle and Finally Detecting the License plate of the vehicle. Detecting the 
License plate and vehicle speed using machine learning is tough but beneficial 
task. For the past few years Convolution Neural Network (CNN) has been 
widely used in computer vision for vehicle detection and identification. Dlibs 
are used to track the multiple objects at the same time. 
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I. INTRODUCTION 

Vehicle speed estimation and License plate detection is one 
of the most important concepts in traffic controlling systems. 
Since, speed limit violation needs to be control and thus 
prevent drivers from excessive speed. Also, Road accidents 
cases can be decreased and we can try to make the road 
safer with the stringent traffic rules. Main objective is to 
identify the over speed vehicles, then automatic detection of 
the License plate and drivers are getting fined. Currently 
using Radar concepts which need manual intervention to 
detect the over speed vehicles. However with the latest 
growing technology we can identify the over speed vehicles 
without manual intervention. Computer Vision and Machine 
learning technologies helps to automatically detect the 
license plate of over speed vehicles and drivers can be fined. 
Even though, various methods and techniques have been 
proposed to estimate the speed of vehicles, there still remain 
some limitations, and thus, studies on detecting the license 
plate and speed estimation still continue. Taking the video 
from surveillance camera is much easier and cost effective. 
Hence considering the costand maintenance, video cameras 
plays a good role to detect the over speed vehicles and 
automatically detect the License plate of the vehicle. 

With the conventional method of artificial observation 
alarming, this method has advantages detection has 
advantages of lower manual intervention, fast reaction, high 
detection rate, treatment and responsibility identification for 
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post-accident since it needs to be addressed and to make a 
road safer with the use of fast growing technology, research 
on this filed has the high demand. In this paper vehicles are 
identified using computer vision technique and machine 
learning algorithms, for automatic License plate detection 
and speed of the vehicle. 

II. RELATED WORK 

There are many studies with the identical topic for 
estimating speed of the vehicle based on image processing 
and automatic License plate detection using machine 
learning algorithms. Survey 1: Vehicle Speed Estimation 
Algorithm Based on DTW Approach -IEEE SENSORS 
JOURNAL, VOL. 17, NO. 8, APRIL 15, 2017 [4].Experimental 
results show it has achieved 96% of accuracy. DTW is the 
most significant expanse for time series study. But In DTW, 
heavy burden in computational analysis and it is necessary 
to find the finest time alignment path. Also DTW is Quadratic 
Complexity - Performance is directly relative to the square 
size of the input data set. Survey 2: A Novel Motion Plane- 
Based Approach to Vehicle Speed Estimation [5].Here 
Centerior of License plate is considered for the speed 
estimation and displacement is calculated for each frame to 
determine the speed of the vehicle. 

3D position of the License plate is considered for the speed 
estimation. And problem is Time consuming as Template 
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matching procedure applicability is limited and tedious for 
the big image patterns. Survey 3: An efficient license plate 
recognition system using convolution neural networks- 2018 
IEEE International Conference on Applied System Invention 
(ICASI) [6] .Advantages is Conventional Neural network 
applied for detecting and recognising the License plate even 
for the blurred images.. Drawbacks are this approach is not 
used for the commercial purpose on Roadways for the 
moving vehicle. 

III. SYSTEM ARCHITECTURE 

In formal system, the speed estimation of the vehicle can be 
taken from different sensors such as camera sensor, acoustic 
sensor, loop detector, ultrasonic sensor etc. In this paper 
speed of the vehicle and License plate of the corresponding 
vehicle can be detected using the videos that taken from 
surveillance camera. 



A. Input Video and Image Acquisition 

Mostly each main roads has the surveillance camera and it 
can be calibrated to certain distance which can capture the 
moving vehicles, those videos can be taken as the input. It 
quite easy and nowadays it's been installed in most of the 
houses. The initial stage of video processing in the computer 
vision is the image acquisition stage. Once the video has been 
has been processed, it makes the videos with number of 
frames. Also there are number of methods in which 
processing is applied to the image to carry out the many 
different vision tasks that are required. Image Processing 
includes the stages such as 

1. Pre-processing, 

2. Back ground subtraction, 

3. Smoothing and 

4. Shadow removal. 


However, if the required image has not been acquire suitably 
then the intentional tasks need to be repeated. Speed is a 
measure of the moving objects and these objects in multiple 
frame needs to be tracked in each frame to identify the 
speed. Hence, to estimate the speed of the vehicle, video is 
broken into frames, then the vehicle being travelled from 
one frame to a new frame is traced out. Distance travelled 
and time taken to move from one pixel to another is 
estimated to determine the speed of the vehicle. A key 
benefit of a digital image processing is that having numeral 
of copy and loss data of very fewer. Extracting sequence of 
images from a video, then analysis them using cv2 (Open 
Source Computer Vision) library 

B. Machine Learning - Haar Classifier: Vehicle 
Identification 

Vehicle classification or Identification refers to the computer 
or software systems is able to trace vehicle in an 
image/scene and recognize each vehicle if we have multiple 
image in each frame. Object detection has plays a very 
important role in identifying the objects in the given image 
such as face detection,, pedestrian counting, vehicle 
detection security systems, web images, and driverless cars. 
These object detection has been used in many applications 
with fast growing technology. Like other Technologies, it has 
the wide uses and amazing creative applications has been 
adopted. Here such object detection of vehicles is done with 
the help of Machine learning algorithm named as Haar 
cascade classifier. Steps for identifying the objects are 

1. Collecting data - Positive and Negative images. 

2. Training system 

3. Validating Result 

Collecting Data 

The initial stage of Haar classifier algorithm is the gathering 
of data set and here we may need to train the system with lot 
of positive images and negative images. Here Positive images 
such as vehicles and Negative Images are other than 
Vehicles. 

> Vehicle 

> Non Vehicle Images. 

It is required to assemble thousands of images i.e. Positive 
Images - True images of vehicle which needs to be 
recognized by the system. Negative Images - False Images, 
which system wants to omit. Applying the positive and 
Negative Images to the system makes it to identify which is 
vehicle and which is non - vehicle. 

Vehicle detection in Haar cascade classifier has 4 stages: 

> Selection of objects to be identified 

> Modelling Integral Images 

> Adaptive boosting Training 

> Cascading Classifiers for multiple detect : Cascade 
Classifier and detect Multi Scale 

With the training set, system is slowly learnt the true and 
false objects. With the validation set, system can be tested to 
recognize the true and false images. 
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Fig 1-Positive Images 
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Fig 2- Negative Images 


Capturing of all vehicles in the frame is always a critical and 
challenging task. Hence we need to train the system with the 
lots of images with training set and validating set. It uses 
Haar features for vehicle detection, which is similar to face 
recognition. Haar cascade classifiers is one of the effective 
object detection method which is proposed by Viola and 
Jones. 

A Haar Cascade is fundamentally a classifier used to detect 
the certain object from the given image. The Haar Cascade is 
trained by superimposing the true image over a set of false 
images. It is generally done on the server with the number of 
stages. Best results with good accuracy can be achieved by 
quality of image set and number of stages. Results achieved 
in this method is reasonably prompt and successful in 
recognizing cars with the videos such as from surveillance 
camera or cctv footages. 

C. Tracking of Vehicle 

Here Tracking refers to each and every vehicle is identified 
and track in it each frame to determine the speed of the 
vehicle. It can be achieved by using the correlation tracker in 
python using dlibs function. The main goal is here to track 
the multiple vehicle at the same time by assigning Tracking 
ID of ach vehicle and keep monitoring how it moves from 
one frame to another in each frame. Speed is estimated by 
calculating the object distance moved from one frame to 
another, i.e object travelled from one pixel to another pixel in 
each frame. This speed calculation is iterated and average of 
speed within certain speed is taken and reported as the 
speed of the vehicle. Hence for every two frames speed has 
been calculated and stored internally, then the average 
speed is calculated and we can return the average speed of 
the vehicle within ROI. Similarly when the object is moved 
out the region of Index, the Correlation ID, tracking process 
is withdrawn and tracking ID is destroyed.. So multiple 
tracking of same vehicle can be avoided. Hence this phase 
mainly aims to assign an ID to the object, track the object in 
each frame, calculate the distance travelled in each frame 


with ROI and eliminates the tracking ID once it moves out of 
ROI. At the end we may have the average speed of the vehicle 
can be estimated. Dlibs is one the library function in python 
helps to track the multiple objects at the same time. 

D. Speed Estimation 

In General speed is estimated by calculating the distance 
travelled of the same object from one frame to another. The 
usual method of using Euclidean distance calculation is used 
and then calculate the pixels per meter moved by the vehicle. 
Here we have started our calculation using the WIDTH and 
HEIGHT of the video is taken and adjusted. We considered 
these plays the important role in estimation speed of the 
vehicle.. The formal method of the Euclidean distance which 
is used to calculate the speed of the vehicle, let Cn(a,b) and 
Cn+l(c,d) is point of the object in frame n and n + 1 
respectively. 

The distance dl is calculated by Euclidean distance is 
mentioned below 

^ 

PPM refers to pixels per meter. This value is estimated 
manually using the WIDTH and HEIGHT of the video that 
varies in each video taken from the camera. Hence values 
may be vary from each video and that needs to adjusted 
manually. Video processing helps to identify these values. 
Also WIDTH of the road plays the important role in speed 
estimation of the vehicle. In the above Euclidean calculation, 
d_pixels is the distance moved by the vehicle from one frame 
to another. So for the standard conversion i.e. it would need 
to calculate the distance in meter say d_meter which is 
calculated as follows 

D_meters = d_pixels / pixels per meter 

Another interesting factor is Frames per second which is 
how many frames can be obtained in a second. So by 
knowing these values, speed of the vehicle can be estimated 
in Km/Hr. 

Speed = D_meters * frames per second *3.6 

which gives the speed of the vehicle in km/hr (3.6 is standard 
conversion of converting meter into kilometre). 

Each object is tracked using the correlation tracker and 
bounding box is applied on the vehicle inside the ROI. 
Average speed of the Vehicle is displayed over the bounding 
box. So the output video has the speed mentioned and 
displayed over the each vehicle. 

E. License Plate Recognition. 

License Plate Recognition makes use of concept of optical 
character recognition to read the number plate characters 
one by one. Simply Images of the vehicle is given as Input 
and ALPR reads the characters using Machine learning 
Algorithms. Using this machine learning technique accuracy 
can be improved by training process, i.e mapping a character 
like image to its real characters and distinguish As, Bs etc. 

LPR is also called ALPR (Automatic License Plate 
Recognition) that consists of 3 major stages. 1. Detecting the 
License Plate. 2. Character Segmentation. 3. Character 
Recognition. 
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Approach as follows: 

1. Detect the contours of the image. 

2. Find the Bounding rectangle of all contours. 

3. Compare the validate the height of the License plate 
ratio. 

4. Apply the image segmentation to find the characters in 
it. 

5. Using OCR method with KNN and SVC model under 
supervised learning, recognize the characters in the 
license plate. 

a. Detecting License Plate 

In this stage License plate of the vehicle is detected. But its 

particularly difficult to detect the position, too many styles 

and pattern. 



0 50 100 150 200 250 


Using Machine learning Classification Algorithms, it's quite 
possible to detect the plate from the image of the vehicle . 
Aim is to crop the Regions of the original image with good 
classification results are retained and others are discarded. 
Thus region detection and image classification algorithm 
plays a vital role in detecting the License plate of the vehicle. 
Connected Component analysis approach helps to group and 
label the connected region. 

b. Character segmentation 

Character segmentation is the critical in OCR Process. This 
stage helps to mapping the characters in the license plate 
using CCA and segmented into Individual images. From the 
cropped image of the License plate, characters are segmented 
and each character is resized into 20x20 pixels as separate 
image. The plate_like_objects is referred to the list of region 
that are similar to the license plate region. From the image, 
three regions were identified as candidates for a license plate. 
This process helps to identify the actual content of license 
plate image and eradicate the other regions. Then a 
Connected component analysis was done on the license plate 
and each character image is resized to 20px * 20px. From the 
segmented character image, system can do the recognizing 
the character process. 

To keep track the order of the image i.e. segmented character 
- column list is introduced in which each image is recognized 
sequentially. Finally it can be sorted and printed in the order 
in which the License plate string can be detected. 
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c. Character Recognition. 

It is the Final stage of the License plate detection. Machine 
Learning algorithms are applied here to recognize the 
character in the license plate automatically. Simply it is 
defined as the branch of AI that deals with data and 
processes it to discover pattern that can be used for future 
predictions. The three major classifications of machine 
learning are supervised, unsupervised and reinforcement 
learning. Supervised learning gives the best accuracy as 
system is trained under fixed dataset with expected 
predictions. So for character recognition we know how the 
each character should be. Hence Supervised learning helps 
here. Supervised learning again can be divided into two 
category; classification and regression. Character recognition 
belongs to the classification category K -Nearest Neighbour 
algorithm is used for recognition under supervised learning 
of characters. 


Training data set is provided to choose a supervised learning 
classifier, train a model, test the model and see how accurate 
it is, then use the model for prediction. 



IV. RESULTS AND DISCUSSIONS 

It is well-ordered and commercial than conventional system 
by applying machine learning and Image processing to 
estimate the speed of the moving object and License Plate of 
the vehicle. Speed estimation and License plate detection of 
the vehicle is one the main process in traffic and management 
system, such as delay cost estimation incident detection, flow 
monitoring, etc. 
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This method is comparatively efficient than conventional 
methods such as radar ,loop detector, magnetic poles etc . 
Using camera videos is beneficial and low cost Using 
computer vision and Machine learning technique fast 
developed, this study tries to extract vehicle speed and 
License plate recognition from surveillance video data. This 
paper aims to work out the 2018 AI City Challenge Track 1. 
Three steps are taken: a) multi-object detection, correlation, 
tracking using Faster R-CNN, b) speed estimation of the 
vehicle, and c) Image acquisition and multi tracker of the 
object, d) License plate detection of the vehicle using Machine 
learning algorithms. The results show that by using this 
approach, the speed estimation and License plate number of 
the vehicle can be traced on any roads can be attain a good 
performance. 

Hence this approach using Machine Learning and Image 
Processing technique supports to detect the Over speed 
vehicles automatically with their License plate for each 
vehicle without manual intervention. It can acts as model 
which can be urbanized and complex for larger system to 
build a absolute ITS system for smart city development. 

V. CONCLUSIONS 

Speed estimation and License plate recognition is one of 
many important parts of Intelligent Traffic System (ITS) 
which can be done by using machine learning algorithms. It 
is more proficient and cost-effective than conventional 
technique without using image processing, for example: 
using speed radar and manual inspection of vehicle's 
number plate. A study in this paper is interesting for other 
researches which are discussed the same subject in which 
this paper shows that there is an influence between speed 
estimation, camera angle, and ROI selection while using 
Euclidean distance. Later License plate has been detected 


automatically using machine learning algorithms that gives 
its best accuracy. So that, this paper can be a reference for 
further researches about speed estimation on moving 
vehicle using image processing and Euclidean distance, then 
License plate recognition. 
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